Meta-Analytic Structural Equation Modeling

Mike Cheung

8-10 Feb 2023 (Day 2, Session 1)

What is MASEM?

Terms used in the literature

Benefits of MASEM to SEM researchers

Benefits of MASEM to meta-analysis researchers

Problems in primary research using SEM

Example 1: Brown and Stayman (1992)

Brown and Stayman (1992)

Example 2: Premack and Hunter (1988)

Premack and Hunter (1988)

Example 3: Norton et al. (2013)

Norton et al. (2013)

Example 4: Murayama and Elliot (2012)

Murayama and Elliot (2012, Figure 1)

Example 5: Hagger et al. (2022)

Hagger et al. (2022, Figure 1)

Example 6: Isvoranu, Epskamp, and Cheung (2022)

Isvoranu, Epskamp, and Cheung (2022, Figure 3)

Key procedures and decisions in conducting MASEM

  1. Identify key research questions, constructs, measurements, and structural equation models: Before conducting the MASEM, researchers have to formulate research questions and identify all the relevant key constructs, measurement models, and structural equation models.
  2. Formulate clear inclusion/exclusion criteria. This step is essential in all meta-analyses, including MASEM, because it provides theoretical justifications on whether the selected studies can be meaningfully combined.
  3. Identify and extract the relevant data, including correlation matrices, sample sizes, and study characteristics (moderators).
  4. There are several approaches to conducting MASEM. Choose an appropriate approach to combine the correlation matrices and fit the structural equation models.

Approaches to MASEM

Jak and Cheung (2020, Table 1)

Univariate approach (Viswesvaran & Ones, 1995)

Viswesvaran and Ones (1995) citations over time

Viswesvaran and Ones (1995) citations across journals

Basic ideas

One (sample) size does not fit all

Problems of treating a correlation matrix as a covariance matrix

A simulation study comparing some of these methods

Accuracy of the test statistics in rejecting the proposed models

Jak and Cheung (2022, test statistics)

Accuracy of the parameter estimates

Jak and Cheung (2022, parameter estimates)

Accuracy of the standard errors

Jak and Cheung (2022, standard errors)

GLS approach

TSSEM approach

Fixed-effects TSSEM

Fixed-effects TSSEM: Stage 1 analysis (1)

Fixed-effects TSSEM: Stage 1 analysis (2)

Fixed-effects TSSEM: Stage 2 analysis (1)

Fixed-effects TSSEM: Stage 2 analysis (2)

Fixed-effects TSSEM: Stage 2 analysis (3)

Fixed-effects TSSEM: Stage 2 analysis (4)

Fixed-effects TSSEM: Stage 2 analysis (5)

## Load the libraries
library(metaSEM)
library(symSEM)

## Specify the model
cfa <- "f =~ a*x1 + b*x2 + c*x3
        f ~~ 1*f            ## Fix the factor variance at 1 for identification
        x1 ~~ e11*x1        ## Label the error variances
        x2 ~~ e22*x2
        x3 ~~ e33*x3"

## Plot the model
plot(cfa, color="yellow")

## Convert it to RAM specification
## We will introduce the RAM specification in later.
RAM <- lavaan2RAM(cfa, obs.variables=c("x1", "x2", "x3"))

## Print the model-implied covariance matrix
impliedS(RAM, corr=FALSE)
## $Sigma
##    x1        x2        x3       
## x1 "e11+a^2" "a*b"     "a*c"    
## x2 "b*a"     "e22+b^2" "b*c"    
## x3 "c*a"     "c*b"     "e33+c^2"
## 
## $mu
##   x1 x2 x3
## 1  0  0  0
## 
## $corr
## [1] FALSE
## Print the model-implied correlation matrix
impliedS(RAM, corr=TRUE)
## $Sigma
##    x1    x2    x3   
## x1 "1"   "a*b" "a*c"
## x2 "b*a" "1"   "b*c"
## x3 "c*a" "c*b" "1"  
## 
## $mu
##   x1 x2 x3
## 1  0  0  0
## 
## $corr
## [1] TRUE

  1. Cheung, M. W.-L. (2021). Meta-analytic structural equation modeling. In Oxford Research Encyclopedia of Business and Management. Oxford University Press. https://doi.org/10.1093/acrefore/9780190224851.013.225↩︎

  2. National Research Council (1992). Combining information: Statistical issues and opportunities for research. Washington, D.C.: National Academy Press.↩︎

  3. Brown, S. P., & Stayman, D. M. (1992). Antecedents and consequences of attitude toward the ad: A meta-analysis. Journal of Consumer Research, 19, 34-51.↩︎

  4. Premack, S. L., & Hunter, J. E. (1988). Individual unionization decisions. Psychological Bulletin, 103, 223-234.↩︎

  5. Norton, S., Cosco, T., Doyle, F., Done, J., & Sacker, A. (2013). The Hospital Anxiety and Depression Scale: A meta confirmatory factor analysis. Journal of Psychosomatic Research, 74(1), 74-81.↩︎

  6. Murayama, K., & Elliot, A. J. (2012). The competition-performance relation: A meta-analytic review and test of the opposing processes model of competition and performance. Psychological Bulletin, 138(6), 1035-1070. http://doi.org/10.1037/a0028324↩︎

  7. Hagger, M. S., Cheung, M. W.-L., Ajzen, I., & Hamilton, K. (2022). Perceived behavioral control moderating effects in the theory of planned behavior: A meta-analysis. Health Psychology, 41(2), 155–167.↩︎

  8. Epskamp, S., Borsboom, D., & Fried, E. I. (2018). Estimating psychological networks and their accuracy: A tutorial paper. Behavior Research Methods, 50(1), 195–212.↩︎

  9. Epskamp, S., Isvoranu, A.-M., & Cheung, M. W.-L. (2022). Meta-analytic Gaussian network aggregation. Psychometrika, 87(1), 12–46.↩︎

  10. Isvoranu, A.-M., Epskamp, S., & Cheung, M. W.-L. (2021). Network models of posttraumatic stress disorder: A meta-analysis. Journal of Abnormal Psychology, 130(8), 841–861.↩︎

  11. Viswesvaran, C., & Ones, D. S. (1995). Theory testing: Combining psychometric meta-analysis and structural equations modeling. Personnel Psychology, 48(4), 865-885.↩︎

  12. Becker, B. J. (1992). Using results from replicated studies to estimate linear models. Journal of Educational Statistics, 17(4), 341-362.↩︎

  13. Cheung, M. W.-L. (2014). Fixed- and random-effects meta-analytic structural equation modeling: Examples and analyses in R. Behavior Research Methods, 46(1), 29-40. http://doi.org/10.3758/s13428-013-0361-y↩︎

  14. Cheung, M. W.-L., & Chan, W. (2005). Meta-analytic structural equation modeling: A two-stage approach. Psychological Methods, 10(1), 40-64. http://doi.org/10.1037/1082-989X.10.1.40↩︎

  15. Jak, S., & Cheung, M. W.-L. (2020). Meta-analytic structural equation modeling with moderating effects on SEM parameters. Psychological Methods, 25(4), 430–455. https://doi.org/10.1037/met0000245↩︎

  16. Ke, Z., Zhang, Q., & Tong, X. (2019). Bayesian meta-analytic SEM: A one-stage approach to modeling between-studies heterogeneity in structural parameters. Structural Equation Modeling: A Multidisciplinary Journal, 26(3), 348–370. https://doi.org/10.1080/10705511.2018.1530059↩︎

  17. Cudeck, R. (1989). Analysis of correlation matrices using covariance structure models. Psychological Bulletin, 105(2), 317-327.↩︎

  18. Cudeck, R. (1989). Analysis of correlation matrices using covariance structure models. Psychological Bulletin, 105(2), 317–327. https://doi.org/10.1037/0033-2909.105.2.317↩︎

  19. Jak, S., & Cheung, M. W.-L. (2020). Meta-analytic structural equation modeling with moderating effects on SEM parameters. Psychological Methods, 25(4), 430–455. https://doi.org/10.1037/met0000245↩︎

  20. Jak, S., & Cheung, M. W. (2022, March 16). Can findings from meta-analytic structural equation modeling in management and organizational psychology be trusted?. https://psyarxiv.com/b3qvn.↩︎

  21. Cheung, M. W.-L., & Chan, W. (2005). Meta-analytic structural equation modeling: A two-stage approach. Psychological Methods, 10(1), 40–64. https://doi.org/10.1037/1082-989X.10.1.40↩︎

  22. Cheung, M. W.-L. (2014). Fixed- and random-effects meta-analytic structural equation modeling: Examples and analyses in R. Behavior Research Methods, 46(1), 29–40. https://doi.org/10.3758/s13428-013-0361-y↩︎

  23. Joreskog, K. G., & Sorbom, D. (1996). LISREL 8: A user-s reference guide. Chicago, IL: Scientific Software International, Inc.↩︎

  24. Muthen, B., Kaplan, D., & Hollis, M. (1987). On structural equation modeling with data that are not missing completely at random. Psychometrika, 52(3), 431-462. http://doi.org/10.1007/BF02294365↩︎

  25. Cheung, M. W.-L., & Chan, W. (2009). A two-stage approach to synthesizing covariance matrices in meta-analytic structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 16(1), 28-53. http://doi.org/10.1080/10705510802561295↩︎

  26. Bentler, P. M., & Savalei, V. (2010). Analysis of correlation structures: Current status and open problems. In S. Kolenikov, D. Steinley, & L. Thombs (Eds.), Statistics in the Social Sciences (pp. 1-36). New Jersey: John Wiley & Sons, Inc.↩︎